WebCrawler (synonyms: robot, spider, crawler)

WebCrawler is a search engine that finds references to URLs on the Web and makes a database available for searching. It is a type of spider, but has some major advantages over such engines as the WWW worm; it records URLs for protocols such as gopher, and ftp as well as http; its search pattern is for words rather than strings; and it indexes the contents of documents that it finds as well as the URLs and titles.

URLs:

WebCrawler

The WebCrawler is impressive in its depth of knowledge about those sites it has visited.

WWWW - World Wide Web Worm

The Worm can visit more sites in the same that the WebCrawler is doing a total text search of few sites.

W3E References:

search engines

spider

worm

Print References:

• Que's Computer and Internet Dictionary by Brian Pfaffenberger with David Wall. Que, Indianapolis, IN. 1995. (ISBN: 0-7897-0356-4)
• the World Wide Web Unleashed by John December & Neil Randall. Sams Publishing, Indianapolis, IN. 1995. (ISBN: 0-6723-0737-5)